This file contains the overall goals and context for the project, see Heater profile technical documentation for a more in depth guide to get the code running
The Solar Orbiter magnetometer has two instruments, IBS and OBS, which sit on the boom. The aim is the measure the interplanetary magnetic field, using a fluxgate magnetometer, which is on the order of nT.
This instrument is extremely sensitive, and can measure very small fluctuations in the solar wind. Unfortunately, it can also pick up the magnetic field of the other instruments aboard the spacecraft. Since space is very cold, each instrument has a heater to keep the electronics within a certain temperature window. If the temperature drops below a certain threshold, the heater will turn on. There is a separate code that removes the heater signals from the hundreds of heaters on the spacecraft.
The largest interference signal comes from the magnetometers' own heater, which turns on roughly every 15 minutes for 1 minute at a time. The signal looks like this:
Obviously, an interference on the scale of 1nT is not great when the real solar wind can be on the order of 10nT. You can clearly see how the heater turns on really sharply at 0 seconds, stays on, then turns off sharply at 1 minute.
Radial-Tangential-Normal (RTN) coordinates are defined such that the +X axis points from sun center through the satellite (radial), and the +Z axis points along the projection of the solar North pole (normal). The +Y axis completes the right-handed coordinate system, and points towards the solar West limb (tangential) (source)
Even more frustrating is that fact that the heater profiles change over time, with no obvious cause.
The redder the colour the later the date of the profile
To remove this signal, the current solution takes an average profile from all the 15 minute intervals within a 3 day period. This works reasonably well, however, there are some major issues:
All profiles from 2022, can see that the ones near perihelion (red) are just nonsense
Instead of stepping the window for the average by a day, move it by 15 minutes. This will allow a much smoother change in heater profile, rather than a step change at midnight
Therefore, the goals of this project are to:
To help with these goals, there is a wealth of data about the state of the spacecraft, known as house keeping data. This keeps track of everything that happens on the spacecraft, such as:
and many more...
To better see the systematic changes in heater profiles, we often plot the profiles as a 2D histogram.
The left panel shows the heater profiles, where a redder colour indicates a later profile date. We have taken these profiles and stacked them vertically, equating the value of the component with the colour scale.
To get your eye in, notice that the blue profiles on the left end at 5 minutes (300 seconds), which equals the shorter part of the histogram.
For example, looking at the Bt component (second panel on the right figure). We can see that when the profile length changes from 300 seconds to 800 seconds, there is a decaying signal at around 200 seconds profile time.
This signal persists for 2 weeks, and is thought to be due to residual heat flux after the instrument changes operational temperature range.
The heaters are turned on if the temperature drops below a certain threshold. The MAG team usually runs this at around
This problem sounds like a good use case for supervised machine learning, where the model is trying to learn a known answer.
But how should we set this problem up?
So far, we have tried to learn how to predict the average profiles that are created for each day of data. We question whether there is a better method in this section.
Solar Orbiter was launched in February 2020, therefore we have on the order of 900 heater profiles to learn from.
However, each heater profile runs for around 15 minutes. If we tried to predict just one magnetic field component at 1 second intervals, we need to predict
Therefore, there is probably not enough data (900 days) to train a model to directly predict a vector of size 900 (15 minutes in 1 second intervals).
This final statement might not be true if we changed the averaging method to create a profile for each 15 minute interval instead of each day. Now we would have
Inspired by this kaggle competition, we have turned this problem into a regression problem. By stacking all the profiles together, we are now trying to just predict one y value at a time.
In other words, we are using Time through the profile as a feature to train the model.
This simple perspective shift means we now have
As you will see later, we originally tried to train a separate model for each magnetic field component. However, we then decided that it would make more physical sense to have a single model that predicted 3 outputs.
The problem with previous two methods is that the best they can do is match the average profiles. Therefore, the model will inherit all the same problems from the averaging technique
Instead of learning what the average profiles look like, we could try and predict the shape for all the 15 minutes individually.
Machine learning needs to minimise some value, like the mean square error. Therefore, we just need to devise some function to be minimised... This is easier said than done though, and has not been explored yet.
The rest of this document outlines the steps taken and current outputs of the model, as well as where we could go next.
Again, the technical implementation details are outlined in Heater profile technical documentation
The main challenge with this problem is to train the model to learn from, rather than memorise, the training data
The model is exceptionally good on the trained data, but can hallucinate when used on unseen data. This is because the model can try and memorise the profiles, instead of learning the relationships.
This challenge is the primary motivator behind many of the techniques outlined below.
The current house keeping data used for the model. We have tried to include why we think these features would affect the heater profiles.
The house keeping data are grouped into different files, which is reflected in the subheadings here.
Since we have framed the problem to predict a heater profile per day, we need to also calculate the house keeping data per day, i.e. even though temperature can change many times per day, we can only give the model 1 value per day.
Need to add more house keeping data, however, this creates more features which seemed to confuse the training. We have tried to address this with feature selection
Called Radius in the features, since it comes from describing the position of the spacecraft in longitude, latitude and radius. To see how this is calculated, look in Heater profile technical documentation
Since the heater interference is probably driven by thermal gradients, useful to know the temperature of the instruments
Temperature of the IBS and OBS instruments. Top panel shows the raw data, the second panel is a 1 day average (since we need 1 house keeping data point per day)
3rd panel is the difference between OBS and IBS temperatures
The rest of the features are trying to capture the decaying Bt signal discussed earlier. You can see that the OBS temperatures are either time_since_T_high) and also how long the temperature was run at
This worked pretty well, but we also added another feature (cumulative) to capture how the thermal gradient would reduce over time. Ideally this would decay exponentially, but is a lot easier to just say it is linear for now.
Angle of the solar arrays could reflect sunlight onto the heater
First panel shows the raw data (blue) and the 1 day average (orange)
Second panel tries to capture the change in angle:
The changes in these components are the same as the solar array encoding above
RPW is another instrument on Solar Orbiter that sits in between IBS and OBS.
We have the temperature of its search coil magnetometer (SCM), and the change in this value.
I assume this stands for MAG power and Amp (current), needs checking
The Mag A component had many drop outs (where it went to 0). We have tried to capture this with the No_times_A_off parameter which is the log(number of times off per day)
Shows when the heater turns one, which is normally every 15 minutes
You can see that the heater changed (blue to pink) on 2021-11-08.
The profiles near perihelion are not usable and would hinder the model's training. Therefore, we have removed the times listed in bad_dates.csv (identified by eye)
start,end
2020-10-26,2020-11-06
2021-01-15,2021-03-12
2021-05-29,2021-06-03
2022-02-17,2022-04-20
2022-09-12,2022-11-07
2022-11-07,2022-11-08
2023-03-25,2023-05-05
You may have noticed that the features above look a bit step-like. This is because we have only allowed the features to take a certain number of values, e.g. 0.1 and 0.14 would both be taken as 0.1. The technical term for this is reducing the cardinality.
This helps step overfitting, since the model cannot work out which combination of values corresponds to which day, i.e. only one day might have the value 0.012312041243 while another day had 0.0123014035407564, so the model just memorises the unique values.
This is done by helpers.reduce_cardinality(features, feature_name, N, bins)
Perhaps the change in profile is because something happened in the previous day. To capture this, we have shifted the data and saved in a new column Feature-name 1 where 1 means 1 day in the past.
When researchers are coming up with these machine learning algorithms, they often use some basic assumptions, such as the data is a Gaussian centred on 0. Therefore, we have to scale our data to match their predictions. This is easily done in sklearn.
To save computing time, I downsample the heater profiles when learning. When making the models we have two parameters to control the time steps:
ds_duration for the length of time through the profile (ds is downsample)ds_period for the difference between time stepsTowards the end of the project, we switched to using custom time steps. This allows us to have a finer resolution in the first minute or so, then only every 5 seconds after that. The exact function for this is helpers.get_var_time_bins(ds_duration)
In the code, we call this by setting ds_period = -1
As mentioned in the calculate of duty cycle, the instrument switched which heater it used on 2021-11-08. We found that the profiles look distinctly different after this switch, so the best results only train on the data after this date.
As we added more features, we noticed the models struggled to train as there was too much noise in the features. Therefore, we now use Recursive Feature Elimination (RFE) which recursively trains a model and removes the least relevant features.
Most of the research was done with LightGBM, since it is lightweight and fast. There is an API you can use to fit well with sklearn.
This is a type of gradient boosted decision tree. In essence, this uses lots of weak leaners (small decision trees) to build a more complex model. In this method, the next tree is generated to minimise the error from all the previous trees.
In contrast, random forest is a collection of randomly generated trees that all vote on the answer collectively.
The main appeal of gradient boosting is that we can control when it stops learning, using a technique called early stopping. If we give the model a completely separate validation set, it will stop learning when the model doesn't improve the score on the validation data within N new trees.
However, this model can only learn to predict one component at a time, which is not ideal when trying to predict a vector in our problem.
Instead, we then opted to use a random forest, which can learn to predict a vector output. Unfortunately, this does not have the early stopping capability, and instead you have to try and stop overfitting by tweaking the model hyperparameters.
This document has hopefully outlined the general ideas behind the project. See Heater profile technical documentation for a more in depth guide to get the code running
.
.
extra space to fix a glitch in the html...